Our research is in the area of computational interaction. We use methods of optimization, machine learning, and mathematical modeling to understand and improve our interaction with computers. In particular, our work centers around the following topics:
Optimal adaptation of interfaces
We investigate how to optimally adapt user interfaces at run-time to changes in the usage context, such as the user’s environment, their cognitive load, current task, or personal interaction strategy. On the one hand, this includes formulating optimization models for adapting interfaces. On the other hand, we develop methods to identify changes in the user’s context and models to predict how such changes impact the user’s behavior. Example projects include:
- Towards Flexible and Robust User Interface Adaptations With Multiple Objectives. In: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, Association for Computing Machinery, San Francisco, CA, USA, 2023, ISBN: 9798400701320.
- AUIT – the Adaptive User Interfaces Toolkit for Designing XR Applications. In: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST'22, 2022.
- XRgonomics: Facilitating the Creation of Ergonomic 3D Interfaces. In: Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI'21), ACM, 2021.
- Detecting Relevance during Decision-Making from Eye Movements for UI Adaptation. In: Symposium on Eye Tracking Research and Applications, Association for Computing Machinery, 2020, ISBN: 9781450371339.
- Context-Aware Online Adaptation of Mixed Reality Interfaces. In: Symposium on User Interface Software and Technology, ACM, 2019, ISBN: 9781450368162.
- AdaM: Adapting Multi-User Interfaces for Collaborative Environments in Real-Time. In: SIGCHI Conference on Human Factors in Computing Systems, ACM, 2018.
Human-AI interaction and oversight
Our recent work focuses on understanding, modeling, and supporting people in their interaction with AI systems. On the one hand, we consider contexts that involve AI systems supporting users in their tasks (e.g. intelligent text input systems, AI-supported decision making). On the other hand, we are also interested in supporting humans in overseeing highly automated systems. We study how users rely on and adapt to AI support (e.g. from their gaze behavior), how AI support affects people’s perceived responsibility and autonomy, and how interface design can shape human motivation, perceived autonomy, and trust.
- RelEYEance: Gaze-Based Assessment of Users’ AI-reliance at Run-Time. In: Proc. ACM Hum.-Comput. Interact., vol. 9, no. ETRA16, 2025.
- Typing Behavior is About More than Speed: Users' Strategies for Choosing Word Suggestions Despite Slower Typing Rates. In: Proc. ACM Hum.-Comput. Interact., vol. 7, no. MHCI, 2023.
Computational design of user interfaces
We develop optimization methods that can be used to automate (parts of) the design process of user interfaces. In addition to developing optimization models that capture the problem of UI design, we also focus on methods and tools that facilitate participatory optimizaton to enable stakeholders at all levels to take part in the design process. Interactive optimization tools can help them to learn about the problem and broadly explore possible solutions. They also make assumptions and trade-offs more explicit and help to objectively assess individual’s expectations and ideas.
See for example the following works:
- AZERTY Amélioré: Computational Design on a National Scale. In: Communications of the ACM, vol. 64, no. 2, 2021, ISSN: 0001-0782.
- Computational Support for Functionality Selection in Interaction Design. In: ACM Transaction on Computer-Human Interaction, vol. 24, no. 5, 2017, ISSN: 1073-0516.
- Assignment Problems for Optimizing Text Input. Aalto University, 2018.
- Towards Multi-Objective Optimization for UI Design. Workshop on Principles, Techniques and Perspectives on Optimization and HCI, CHI'15, Seoul, Korea, 2015.
Understanding and predicting user behavior
Our work builds on empirical studies to capture users’ behavior in interaction with different interfaces. This includes planning and conducting studies, analyzing data, and transforming empirical observations into predictive models using machine learning, statistical modeling, and other techniques. The results are actionable insights that can be used for optimization and adaptation or implemented as part of design tools. See for example the following projects:
- Shifting Focus with HCEye: Exploring the Dynamics of Visual Highlighting and Cognitive Load on User Attention and Saliency Prediction. In: Proceedings of the ACM on Human-Computer Interaction, vol. 8, No. ETRA , no. 236 , 2024.
- Typing Behavior is About More than Speed: Users' Strategies for Choosing Word Suggestions Despite Slower Typing Rates. In: Proc. ACM Hum.-Comput. Interact., vol. 7, no. MHCI, 2023.
- XRgonomics: Facilitating the Creation of Ergonomic 3D Interfaces. In: Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI'21), ACM, 2021.
- Complex Interaction as Emergent Behaviour: Simulating Mid-Air Virtual Keyboard Typing using Reinforcement Learning. IEEE Transactions on Visualization and Computer Graphics, IEEE, 2021.
- How We Type: Movement Strategies and Performance in Everyday Typing. In: SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, US, 2016, ISBN: 978-1-4503-3362-7.
- Investigating the Dexterity of Multi-Finger Input for Mid-Air Text Entry. In: SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, New York, USA, 2015, ISBN: 9781450331456.